Scalable High Performance Dimension Reduction

Dimension reduction is a useful tool for visualization of such high-dimensional data to make data analysis feasible for such vast volume and high-dimensional scientific data. Among the known dimension reduction algorithms, multidimensional scaling algorithm is investigated in this proposal due to its theoretical robustness and high applicability. Multidimensional scaling is known as a non-linear optimization problem so that it is easy to be trapped in local optima if EM-like hill-climbing approach is used to solve it. In order to avoid local optima, the author has applied deterministic annealing (DA) approach to the well-known EM-like multidimensional scaling algorithm called SMACOF. Furthermore, the MDS algorithm is necessary to be parallelized to deal with large amount of data via distributed-memory environment, such as multicore cluster systems, since multidimensional scaling requires O(N2) physical memory as well as O(N2) computational resources. Although parallelization enables SMACOF algorithm to deal with tens of thousands or even hundreds of thousands points, it is still difficult to run parallel SMACOF algorithm with millions points, since it requires too much memory and computation to execute. Thus, the author proposes an interpolated approach to utilizing the known mapping of only a subset of the given data, named in-sample data. This approach effectively reduces computational complexity. With minor trade-off of approximation, interpolation method makes it possible to process millions of data points with modest amounts of computation and memory requirement. Since huge amount of data are dealt, the author presents how to parallelize proposed interpolation algorithms, as well. As we expected, the applying DA approach to SMACOF algorithm enables the proposed algorithm not to be stucked in the local optima but to find better results consistently with tested biological sequence data. Also, applying distributed parallelism to SMACOF algorithm helps to run with bigger data size which is not apt to a single compute node. The author is going to compare pure MPI parallel model with hybrid (MPI-Threading) parallel model to aim at finding better parallel model for the SMACOF (and DA-SMACOF) algorithm. Also, the experimental results illustrate that the quality of interpolated mapping results are comparable to the mapping results of original algorithm only. In parallel performance aspect, the interpolation method is parallelized with high efficiency. With the proposed interpolation method, it is possible to construct a configuration of two-million out-of-sample data into the target dimension, and the number of out-of-sample data can be increased further. The affect of the weight function in the STRESS value will also be investigating with several non-uniform weight function as well as uniform weight function.